Exercise 2
This exercise investigates a multidimensional CAT with mirtCAT using off-line components from a previously estimated model. As well, Monte Carlo simulation aspects are constructed to demonstrate the general behavior of the multidimensional CAT.
Question 1
The mirt model is available in the Exercise_02.Rdata workspace file. Read this file into R with load('Exercise_02.Rdata'), and inspect the object workspace.
Given that all the hard estimation work has been done, all you get to do if have fun with mirtCAT! Familiarize yourself with the input objects to get a better understanding of what you’re working by using functions such as coef(), plot(), etc.
library(mirtCAT)## Loading required package: mirt## Loading required package: stats4## Loading required package: lattice## Loading required package: shinyload('Exercise_02.Rdata')
coef(mod, simplify=TRUE)## $items
## a1 a2 d g u
## Item.1 0.850 0.000 -2.732 0.166 1
## Item.2 1.327 0.000 -0.100 0.156 1
## Item.3 1.691 0.000 0.976 0.177 1
## Item.4 0.604 0.000 0.414 0.214 1
## Item.5 1.389 0.000 0.912 0.183 1
## Item.6 1.422 0.000 1.984 0.164 1
## Item.7 1.028 0.000 1.169 0.168 1
## Item.8 1.037 0.000 -0.509 0.236 1
## Item.9 1.031 0.000 0.704 0.169 1
## Item.10 0.935 0.000 -0.198 0.206 1
## Item.11 1.058 0.000 -0.538 0.230 1
## Item.12 0.905 0.000 -2.856 0.196 1
## Item.13 0.968 0.000 -0.790 0.170 1
## Item.14 1.245 0.000 0.488 0.181 1
## Item.15 1.629 0.000 2.168 0.168 1
## Item.16 1.182 0.000 0.501 0.222 1
## Item.17 1.048 0.000 0.620 0.178 1
## Item.18 0.929 0.000 -0.966 0.233 1
## Item.19 0.950 0.000 0.163 0.166 1
## Item.20 2.521 0.000 -2.078 0.164 1
## Item.21 1.272 0.000 0.485 0.187 1
## Item.22 1.054 0.000 0.697 0.183 1
## Item.23 1.070 0.000 0.186 0.168 1
## Item.24 1.402 0.000 0.701 0.140 1
## Item.25 0.992 0.000 0.312 0.183 1
## Item.26 0.791 0.000 0.760 0.162 1
## Item.27 1.451 0.000 1.842 0.217 1
## Item.28 0.898 0.000 1.112 0.156 1
## Item.29 1.216 0.000 0.033 0.218 1
## Item.30 0.922 0.000 -1.114 0.164 1
## Item.31 1.700 -0.016 0.418 0.191 1
## Item.32 1.059 0.077 -0.400 0.211 1
## Item.33 0.987 0.512 1.493 0.167 1
## Item.34 1.051 0.300 -1.607 0.200 1
## Item.35 0.749 -0.149 -0.416 0.180 1
## Item.36 0.860 0.107 0.422 0.151 1
## Item.37 0.635 -0.340 -0.152 0.175 1
## Item.38 0.817 0.263 -0.606 0.236 1
## Item.39 1.118 0.292 -0.305 0.176 1
## Item.40 1.062 0.636 0.630 0.159 1
## Item.41 1.887 0.124 0.895 0.181 1
## Item.42 0.886 -0.142 0.660 0.192 1
## Item.43 0.945 0.020 2.273 0.213 1
## Item.44 1.123 -0.151 1.173 0.219 1
## Item.45 0.906 -0.248 0.288 0.185 1
## Item.46 0.913 0.050 -0.660 0.220 1
## Item.47 0.876 -0.269 2.919 0.161 1
## Item.48 0.839 0.050 0.677 0.123 1
## Item.49 1.044 0.106 -0.684 0.169 1
## Item.50 1.052 -0.016 0.186 0.168 1
## Item.51 0.710 -0.059 -0.324 0.123 1
## Item.52 1.026 -0.195 -0.275 0.200 1
## Item.53 0.876 -0.333 -0.934 0.160 1
## Item.54 0.901 0.255 0.117 0.176 1
## Item.55 1.163 0.007 0.319 0.140 1
## Item.56 1.446 0.249 -1.078 0.199 1
## Item.57 2.002 -0.373 -3.233 0.246 1
## Item.58 0.969 0.051 -0.255 0.158 1
## Item.59 1.977 0.202 0.030 0.179 1
## Item.60 0.863 -0.008 0.594 0.248 1
## Item.61 0.197 1.153 0.059 0.212 1
## Item.62 0.765 0.966 0.413 0.246 1
## Item.63 -0.010 2.265 -1.098 0.182 1
## Item.64 -0.201 1.530 0.711 0.178 1
## Item.65 -0.002 2.111 0.719 0.168 1
## Item.66 0.533 1.251 0.252 0.229 1
## Item.67 -0.342 1.011 1.357 0.147 1
## Item.68 0.410 0.776 0.404 0.192 1
## Item.69 0.399 1.009 0.264 0.181 1
## Item.70 0.101 1.307 0.268 0.259 1
## Item.71 0.002 1.656 0.437 0.165 1
## Item.72 -0.137 1.318 1.060 0.142 1
## Item.73 -0.110 0.859 0.452 0.189 1
## Item.74 0.194 1.493 0.663 0.181 1
## Item.75 0.621 0.745 -1.136 0.159 1
## Item.76 -0.046 1.094 -0.370 0.186 1
## Item.77 -0.417 1.111 1.477 0.174 1
## Item.78 -0.217 0.681 -1.224 0.163 1
## Item.79 0.077 1.610 0.258 0.198 1
## Item.80 -0.095 1.013 0.405 0.173 1
## Item.81 -0.053 1.105 0.976 0.134 1
## Item.82 -0.051 1.856 -0.349 0.116 1
## Item.83 -0.412 1.478 0.159 0.166 1
## Item.84 -0.052 1.182 -1.763 0.230 1
## Item.85 0.255 1.425 0.339 0.209 1
## Item.86 0.209 1.377 -0.667 0.221 1
## Item.87 0.165 2.011 -0.239 0.214 1
## Item.88 -0.121 1.327 -1.188 0.192 1
## Item.89 -0.057 1.422 0.385 0.142 1
## Item.90 -0.358 1.356 0.667 0.189 1
## Item.91 0.000 1.091 -0.305 0.235 1
## Item.92 0.000 1.258 1.825 0.102 1
## Item.93 0.000 1.997 0.671 0.160 1
## Item.94 0.000 0.939 0.949 0.218 1
## Item.95 0.000 1.267 2.049 0.199 1
## Item.96 0.000 1.838 -0.651 0.207 1
## Item.97 0.000 1.138 0.809 0.196 1
## Item.98 0.000 0.890 0.987 0.199 1
## Item.99 0.000 0.941 -0.006 0.183 1
## Item.100 0.000 1.087 0.319 0.157 1
## Item.101 0.000 0.947 -1.012 0.149 1
## Item.102 0.000 1.130 0.470 0.184 1
## Item.103 0.000 1.079 -0.701 0.170 1
## Item.104 0.000 1.156 0.814 0.157 1
## Item.105 0.000 1.380 -0.811 0.195 1
## Item.106 0.000 1.473 0.319 0.187 1
## Item.107 0.000 2.021 -0.847 0.230 1
## Item.108 0.000 1.196 -0.246 0.151 1
## Item.109 0.000 1.109 -1.553 0.167 1
## Item.110 0.000 1.899 0.128 0.180 1
## Item.111 0.000 2.037 0.985 0.144 1
## Item.112 0.000 1.237 0.183 0.174 1
## Item.113 0.000 1.105 -1.766 0.162 1
## Item.114 0.000 0.707 -0.621 0.127 1
## Item.115 0.000 1.865 1.656 0.174 1
## Item.116 0.000 0.950 1.810 0.167 1
## Item.117 0.000 0.872 -1.175 0.219 1
## Item.118 0.000 3.044 -0.367 0.165 1
## Item.119 0.000 1.311 0.354 0.200 1
## Item.120 0.000 1.209 0.319 0.187 1
##
## $means
## F1 F2
## 0 0
##
## $cov
## F1 F2
## F1 1.0 NA
## F2 0.5 1# Expected scoring surface for full test
plot(mod)
# Expected item bundles (each of size 30)
plot(mod, which.items = 1:30)
plot(mod, which.items = 31:60)
plot(mod, which.items = 61:90)
plot(mod, which.items = 91:120)
# Expected information for full test
plot(mod, type = 'info')
# Expected information surface bundles (each of size 30)
plot(mod, type = 'info', which.items = 1:30)
plot(mod, type = 'info', which.items = 31:60)
plot(mod, type = 'info', which.items = 61:90)
plot(mod, type = 'info', which.items = 91:120)
Question 2
Generate 8 independent response patterns according to the population ability sets \(\theta\) = [-1,-1]; [-1, 0]; [-1,1]; [0,-1]; [0,0]; [0,1]; [1,-1]; and [1,1] using the generate_pattern() function.
Theta <- cbind(c(-1, -1, -1, 0, 0, 0, 1, 1),
c(-1, 0, 1, -1, 0, 1, -1, 1))
Theta## [,1] [,2]
## [1,] -1 -1
## [2,] -1 0
## [3,] -1 1
## [4,] 0 -1
## [5,] 0 0
## [6,] 0 1
## [7,] 1 -1
## [8,] 1 1set.seed(1)
patterns <- generate_pattern(mod, Theta)
patterns[, 1:6]## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 0 0 1 0 0 1
## [2,] 0 0 1 0 0 1
## [3,] 0 0 0 0 1 1
## [4,] 1 0 1 1 1 1
## [5,] 0 1 1 1 1 1
## [6,] 1 0 0 1 0 1
## [7,] 1 1 1 1 1 0
## [8,] 0 1 1 1 1 1Question 3
Given the previously generated patterns, pass this object to mirtCAT() to run the multidimensional CAT sessions. Use the selection criteria DPrule throughout (including the initial item), and terminate the test when both ability estimates have \(SE(\hat{\theta}) < .35\).
When completed, plot the results of each response pattern. Finally, create a simple table() indicating how often the items were selected in the response patterns (hint: use the summary() function on each element in the mirtCAT() results, and extract the suitable element).
results <- mirtCAT(mo=mod, local_pattern = patterns, criteria = 'DPrule',
start_item = 'DPrule', design = list(min_SEM = c(.35, .35)))
for(i in 1:length(results)) print(plot(results[[i]]))
ias <- lapply(results, function(x) summary(x)$items_answered)
items_used <- do.call(c, ias)
table(items_used)## items_used
## 2 3 5 6 7 9 11 14 15 16 17 19 20 21 22 23 24 25
## 8 8 8 5 5 5 2 8 5 8 5 2 6 8 5 7 8 4
## 27 28 29 31 32 33 36 39 40 41 44 45 48 49 50 52 55 56
## 5 2 8 8 4 4 2 5 8 8 5 1 2 3 7 4 8 4
## 58 59 61 62 63 64 65 66 67 69 70 71 72 74 75 76 77 79
## 2 8 2 8 6 7 7 6 1 1 3 8 4 6 1 2 2 8
## 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97
## 1 2 7 8 1 8 3 7 3 8 6 1 3 7 1 2 5 1
## 98 100 101 102 103 104 105 106 107 108 109 110 111 112 113 115 116 118
## 1 1 1 1 2 3 3 8 5 3 1 8 7 5 1 5 1 8
## 119 120
## 6 3# item 1 never picked because it a) didn't discriminate as well and b) was too difficult
head(coef(mod, simplify=TRUE)$items)## a1 a2 d g u
## Item.1 0.8503394 0 -2.73221952 0.1657584 1
## Item.2 1.3274088 0 -0.09979059 0.1556896 1
## Item.3 1.6910208 0 0.97603173 0.1771342 1
## Item.4 0.6042850 0 0.41386892 0.2144953 1
## Item.5 1.3892130 0 0.91232216 0.1831313 1
## Item.6 1.4216480 0 1.98373220 0.1638570 1Question 4
The previous question’s simulations were a bit on the slow side, and obviously will grow if more response patterns are studied or more intensive CAT sessions are performed. To circumvent some of these issues, try using the build-in parallelization features for Monte Carlo simulations.
Load the parallel package and define a cluster object using cl <- makeCluster(2). Note that Windows users will likely need to allow permission for OS parallelization if this is the first time you’ve used parallel processing in R. Next, supply the cl object to the syntax used in Question 3, and rerun the simulation.
library(parallel)
cl <- makeCluster(2) # could use detectCores() if more response patterns present
results <- mirtCAT(mo=mod, local_pattern = patterns, criteria = 'DPrule', start_item = 'DPrule',
design = list(min_SEM = c(.35, .35)), cl=cl, progress=TRUE)When done, remove the cluster object with stopCluster(cl).
stopCluster(cl)